A practical Bayesian framework for backpropagation networks

A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible (1) objective comparisons between solutions using alternative network architectures, (2) objective stopping rules for network pruning or growing procedures, (3) objective choice of magnitude and type of weight decay terms or additive regularizers (for penalizing large weights, etc.), (4) a measure of the effective number of well-determined parameters in a model, (5) quantified estimates of the error bars on network parameters and on network output, and (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian "evidence" automatically embodies "Occam's razor," penalizing overflexible and overcomplex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalization ability and the Bayesian evidence is obtained

Bayesian interpolation

Although Bayesian analysis has been in use since Laplace, the Bayesian method of model-comparison has only recently been developed in depth. In this paper, the Bayesian approach to regularization and model-comparison is demonstrated by studying the inference problem of interpolating noisy data. The concepts and methods described are quite general and can be applied to many other data modeling problems. Regularizing constants are set by examining their posterior probability distribution. Alternative regularizers (priors) and alternative basis sets are objectively compared by evaluating the evidence for them. “Occam's razor” is automatically embodied by this process. The way in which Bayes infers the values of regularizing constants and noise levels has an elegant interpretation in terms of the effective number of parameters determined by the data set. This framework is due to Gull and Skilling

Information-based objective functions for active data selection

Learning can be made more efficient if we can actively select particularly salient data points. Within a Bayesian learning framework, objective functions are discussed that measure the expected informativeness of candidate measurements. Three alternative specifications of what we want to gain information about lead to three different criteria for data selection. All these criteria depend on the assumption that the hypothesis space is correct, which may prove to be their main weakness

Poorly connected groups

We investigate groups whose Cayley graphs have poor\-ly connected subgraphs. We prove that a finitely generated group has bounded separation in the sense of Benjamini--Schramm--Tim\'ar if and only if it is virtually free. We then prove a gap theorem for connectivity of finitely presented groups, and prove that there is no comparable theorem for all finitely generated groups. Finally, we formulate a connectivity version of the conjecture that every group of type $F$ with no Baumslag-Solitar subgroup is hyperbolic, and prove it for groups with at most quadratic Dehn function.Comment: 14 pages. Changes to v2: Proof of the Theorem 1.2 shortened, Theorem 1.4 added completing the no-gap result outlined in v

Analysis of Linsker's simulations of Hebbian rules

Linsker has reported the development of center-surround receptive fields and oriented receptive fields in simulations of a Hebb-type equation in a linear network. The dynamics of the learning rule are analyzed in terms of the eigenvectors of the covariance matrix of cell activities. Analytic and computational results for Linsker's covariance matrices, and some general theorems, lead to an explanation of the emergence of center-surround and certain oriented structures. We estimate criteria for the parameter regime in which center-surround structures emerge

Poincar\'e profiles of groups and spaces

We introduce a spectrum of monotone coarse invariants for metric measure spaces called Poincar\'{e} profiles. The two extremes of this spectrum determine the growth of the space, and the separation profile as defined by Benjamini--Schramm--Tim\'{a}r. In this paper we focus on properties of the Poincar\'{e} profiles of groups with polynomial growth, and of hyperbolic spaces, where we deduce a connection between these profiles and conformal dimension. As applications, we use these invariants to show the non-existence of coarse embeddings in a variety of examples.Comment: 55 pages. To appear in Revista Matem\'atica Iberoamerican

Improving the Efficiency of Genomic Selection

We investigate two approaches to increase the efficiency of phenotypic prediction from genome-wide markers, which is a key step for genomic selection (GS) in plant and animal breeding. The first approach is feature selection based on Markov blankets, which provide a theoretically-sound framework for identifying non-informative markers. Fitting GS models using only the informative markers results in simpler models, which may allow cost savings from reduced genotyping. We show that this is accompanied by no loss, and possibly a small gain, in predictive power for four GS models: partial least squares (PLS), ridge regression, LASSO and elastic net. The second approach is the choice of kinship coefficients for genomic best linear unbiased prediction (GBLUP). We compare kinships based on different combinations of centring and scaling of marker genotypes, and a newly proposed kinship measure that adjusts for linkage disequilibrium (LD). We illustrate the use of both approaches and examine their performances using three real-world data sets from plant and animal genetics. We find that elastic net with feature selection and GBLUP using LD-adjusted kinships performed similarly well, and were the best-performing methods in our study.Comment: 17 pages, 5 figure

The Role of Constraints in Hebbian Learning

Models of unsupervised, correlation-based (Hebbian) synaptic plasticity are typically unstable: either all synapses grow until each reaches the maximum allowed strength, or all synapses decay to zero strength. A common method of avoiding these outcomes is to use a constraint that conserves or limits the total synaptic strength over a cell. We study the dynamic effects of such constraints. Two methods of enforcing a constraint are distinguished, multiplicative and subtractive. For otherwise linear learning rules, multiplicative enforcement of a constraint results in dynamics that converge to the principal eigenvector of the operator determining unconstrained synaptic development. Subtractive enforcement, in contrast, typically leads to a final state in which almost all synaptic strengths reach either the maximum or minimum allowed value. This final state is often dominated by weight configurations other than the principal eigenvector of the unconstrained operator. Multiplicative enforcement yields a “graded” receptive field in which most mutually correlated inputs are represented, whereas subtractive enforcement yields a receptive field that is “sharpened” to a subset of maximally correlated inputs. If two equivalent input populations (e.g., two eyes) innervate a common target, multiplicative enforcement prevents their segregation (ocular dominance segregation) when the two populations are weakly correlated; whereas subtractive enforcement allows segregation under these circumstances. These results may be used to understand constraints both over output cells and over input cells. A variety of rules that can implement constrained dynamics are discussed

Imaginaries of the ideal migrant worker : a Lacanian interpretation

The authors acknowledge the ESRC Centre for Population Change RES 62528001 for sponsoring this research.This paper explores the production of ‘ideal’ migrant workers by recruitment agencies in the context of Latvian labour migration to the UK. The fantasies of the ‘ideal’ worker created by recruiters have a particular hold on migrant subjectivity, but they often hide inconsistencies and slippages implicit within the fabric of recruitment discourse and practice. By drawing on the notions of fantasy and desire as developed by Jacques Lacan, this paper analyses the determination of subjectivity in a migration context and explores both unconscious and conscious processes of identification. On the basis of an analysis of drawings sketched by respondents during qualitative interviews conducted in Latvia, it challenges narrower assumptions about migrants’ search behaviour and stable expectations of labour migration, and exposes the split and contested nature of migrant selfhood.It concludes with conceptual observations about the complex process of identification and the unachievable figure of the ‘ideal’ worker.PostprintPeer reviewe

Technologies for Delivery of Proton and Ion Beams for Radiotherapy

Recent developments for the delivery of proton and ion beam therapy have been significant, and a number of technological solutions now exist for the creation and utilisation of these particles for the treatment of cancer. In this paper we review the historical development of particle accelerators used for external beam radiotherapy and discuss the more recent progress towards more capable and cost-effective sources of particles.Comment: 53 pages, 13 figures. Submitted to International Journal of Modern Physics